Reinforcement Learning Approach for Autonomous UAV Navigation in 3D Space

In the last two decades, the rapid development of unmanned aerial vehicles (UAVs) resulted in their usage for a wide range of applications. Miniaturization and cost reduction of electrical components have led to their commercialization, and today they can be utilized for various tasks in an unknown environment. Finding the optimal path based on the start and target pose information is one of the most complex demands for any intelligent UAV system. As this problem requires a high level of adaptability and learning capability of the UAV, the framework based on reinforcement learning is proposed for the localization and navigation tasks. In this paper, Q-learning algorithm for the autonomous navigation of the UAV in 3D space is implemented. To test the proposed methodology for UAV intelligent control, the simulation is conducted in ROS-Gazebo environment. The obtained simulation results have shown that the UAV can reach the target pose autonomously in an efficient way

AUTONOMNOST KRETANjA MOBILNOG ROBOTA -LETELICE ZA RAD NA VISINAMA – SPECIFIČNOSTI KONFIGURACIJE, MODELIRANjE, FUNKCIONALNA APROKSIMACIJA I MAŠINSKO UČENjE OJAČAVANjEM

Roboti koji imaju mogućnost kretanja uz vertikalnu podlogu (engl. wall-climbing), a u koje spadaju i roboti namenjeni posebnim zadacima kao što je čišćenje gabaritnih staklenih površina eksterijera visokih zgrada (engl. glass and façade-cleaning), predstavljaju predmet brojnih istraživanja u prethodnih trideset godina. Interesovanja naučne i stručne zajednice u ovoj oblasti su posledica pre svega njihovog velikog potencijala za rešavanje mnogobrojnih izazova kao što su održavanje i inspekcija građevinskih konstrukcija, ispitivanja teško dostupnih ili veoma opasnih radnih mesta i dr. Iako je na ovom polju ostvaren vidan napredak poslednjih nekoliko godina, ova tehnologija i dalje poseduje određena ograničenja kao što su nemogućnost kontinualnog kretanja ovih robota po fizički odvojenim površinama visokih zgrada i nemogućnost kretanja po neravnim površinama. Male bespilotne letelice (MBL) predstavljaju aktuelni tip letelica poslednjih dvadeset godina, a koje mogu da se korisno upotrebe u širokom opsegu primena. Minijaturizacija i smanjenje troškova električnih komponenti doveli su do njihove komercijalizacije i masovne upotrebe u mnogim oblastima, kao što su gašenje požara, inspekcija, nadzor, farbanje, održavanje vetro-generatora i brodova. Najveću primenu u praksi pronašli su kvadkopteri, pre svega zbog činjenice da se pomoću četiri veličine, tj. upravljanjem brzinama obrta propelera, može ostvariti šest stepeni slobode pri kretanju. Prednosti ovih letelica su dobre manevarske sposobnosti i jednostavno upravljanje. Međutim, smanjenje njihovih dimenzija dovodi do smanjenja efikasnosti, kao i povećanja viskoznih efekata što je posledica malih Rejnoldsovih brojeva. Koncept predstavljen ovim predavanjem po pozivu podrazumeva sledeće prioritete: 1. Realizovati robotski sistem za čišćenje koji treba da omogući kvalitetno obavljanje ove namene. 2. Ostvariti bezbednu, pouzdanu i laku tranziciju. 3. Osmisliti sistem za prianjanje koji će omogućiti robotu potrebnu mobilnost u svim pravcima pri izrvšavanju zadatka na radnim površinama različitih oblika uz minimalni utrošak energije. 4. Obezbediti robotu mogućnost savladavanja prepreka malih dimenzija radi izbegavanja potrebe za čestom tranzicijom. 5. Ostvariti bezbednost u radu i u slučaju neplaniranog otkaza sistema za prianjanje. 6. Omogućiti bezbedno spuštanje robota na tlo usled otkaza nekog od motora namenjenih generisanju vučne sile. Zadatak koji se postavlja pred mobilni robot letelicu je učenje optimalne putanje od početnog položaja (predstavlja položaj robota u neposrednoj blizini objekta) do ciljnog položaja predstavlja položaj robota na radnoj površini. Određivanje ciljnog položaja se ostvaruje aktiviranjem kamere kao spoljašnjeg senzora koja uz pomoć metoda mašinskog gledanja omogućava mobilnom robotu letelici detektovanje staklenih površina, kao i izdvajanje karakterističnih objekata. Kako je u okviru navedenog problema nemoguće odrediti tačan matematički model kretanja mobilnog robota letelice, odnosno okruženja, uvodi se pojam mašinskog učenja ojačavanjem (engl. Reinforcement Learning). Dva osnovna činioca koncepta mašinskog učenja ojačavanjem predstavljaju inteligentni agent i okruženje. Inteligentni agent ima zadatak da istraživanjem okruženja pomoću eksternih senzora kao i eksploatacijom stečenog znanja generiše optimalno ponašanje u cilju izvršavanja postavljenog tehnološkog zadatka. Stanje sistema se može definisati kao skup komponenti nastalih kao rezultat akcija robota i/ili rezultat interakcije robot okruženje. Stanja mogu činiti sledeći elementi: položaj robota u okruženju u odnosu na izabrani referentni koordinatni sistem, njegova brzina kretanja, položaj i brzina kretanja pokretnih objekata u okruženju i sl. Nagradna oceana stanja (engl. Reward ) se definiše kao stepen uspešnosti odabrane akcije u prethodnom stanju sistema izražene u vidu numeričke vrednosti. Cilj agenta jeste da pronađe skup najpovoljnijih akcija u svim stanjima mobilnog robota letelice pri njenom kretanju od početnog do ciljnog položaja. Na osnovu informacija o trenutnom stanju sistema kao i trenutnom stepenu obučenosti, inteligentni agent treba da odabere akciju koja će ga dovesti u naredni položaj. Ovaj položaj predstavlja ulaz u upravljačku jedinicu za kontrolu položaja koja treba da odredi signale na ulazu u kontrolere rada električnih motora kako bi se generisale odgovarajuće vučne sile. Stanje sistema u razmatranom primeru predstavlja položaj mobilnog robota letelice u režimu lebdenja. Okruženje je sastavljeno od sfera jednakih prečnika, čiji su centri moguća stanja sistema. U konkretnom primeru razmatrani prostor je dimenzija 9000x6000x4000mm, iOvo predavanje po pozivu realizovano je prevashodno za studentsku populaciju tehničkih fakulteta u okviru njihovog Kongresa pod nazivom "Tehnologije modernog inženjerstva", održanog na Zlatiboru 2021

Reinforcement Learning Approach for Autonomous UAV Navigation in 3D Space

In the last two decades, the rapid development of unmanned aerial vehicles (UAVs) resulted in their usage for a wide range of applications. Miniaturization and cost reduction of electrical components have led to their commercialization, and today they can be utilized for various tasks in an unknown environment. Finding the optimal path based on the start and target pose information is one of the most complex demands for any intelligent UAV system. As this problem requires a high level of adaptability and learning capability of the UAV, the framework based on reinforcement learning is proposed for the localization and navigation tasks. In this paper, Q-learning algorithm for the autonomous navigation of the UAV in 3D space is implemented. To test the proposed methodology for UAV intelligent control, the simulation is conducted in ROS-Gazebo environment. The obtained simulation results have shown that the UAV can reach the target pose autonomously in an efficient way

Short term exposure to ethyl pyruvate has long term anti-inflammatory effects on microglial cells

Ethyl pyruvate (EP) has been increasingly appreciated as an anti-inflammatory and neuroprotective agent with potent pharmacological properties relevant for treatment of various CNS disorders. Microglial cells seem to be particularly sensitive to its effects. In this study, microglial cells were exposed to EP for relatively short periods (10-120 min) and inflammatory properties of the cells were determined after 24 h of cultivation. Application of EP in the short-term periods inhibited production of interleukin-6, tumor necrosis factor and nitric oxide in microglial cells. At the same time, the effects on cell viability, reactive oxygen species generation and expression of F4/80 and CD40 of microglial cells were minor. NFkB activation was not affected by EP in the cells during the short exposures, thus implying that the observed effect of EP on cytokine and nitric oxide generation was performed in NFkB independent way. Importantly, effects of the short term EP treatment on microglial cells were detected by a real time cell analysis, as well. The observed ability of EP to affect microglial cell function after relatively short time of exposure is relevant for its therapeutic potential against inflammatory disorders of the CNS. (C) 2015 Elsevier Masson SAS. All rights reserved.Ministry of Education, Science and Technological Development of the Republic of Serbia {[}OI173035, OI173013

Mg/Cu co-substituted hydroxyapatite – Biocompatibility, mechanical properties and antimicrobial activity

The aim of this study was to improve the mechanical properties and to optimize antimicrobial activity of hydroxyapatite (HAP) by simultaneous doping with Mg and Cu ions in order to obtain material that would be able to assist in the bone/tooth healing process, prevent post-implementation infections and provide satisfying values of hardness and fracture toughness for biomedical application. Ion doping was done during the hydrothermal synthesis of HAP powders, whereby the content of Mg ions in the starting solution was varied between 1-20 mol. % with regard to Ca ions, while the amount of Cu ions was kept constant at 0.4 mol. %. The green compacts were sintered for 2 h at temperatures ranging 750–1200 °C depending on the Mg content, chosen in agreement with dilatometry results. Presence of Mg ions was found to favour transition from HAP to β−tricalcium phosphate phase (β−TCP), which enabled formation of biphasic HAP/β−TCP and pure β−TCP phase at 160 °C during hydrothermal synthesis. In vitro investigation of antimicrobial activity against Escherichia coli, Staphylococcus aureus and Enterococcus faecalis showed satisfactory antimicrobial activity. MTT assay performed on MRC-5 and L929 cell lines showed excellent cytocompatibility and cell proliferation. Maximum hardness by Vickers and fracture toughness values, 4.96 GPa and 1.75 MPa m1/2 respectively, were obtained upon addition of 5 mol. % Mg, as a consequence of the lowest grain size and porosity, as well as the highest densification rate. This is, to the best of our knowledge, the highest fracture toughness for HAP or β-TCP ceramics reported thus far.This is the peer-reviewed version ofthe article: Veljović, Đorđe, Matić, Tamara, Stamenić, Tanja, Kojić, Vesna, Dimitrijević Branković, Suzana, Lukić, Miodrag J., Jevtić, Sanja, Radovanović, Željko, Petrović, Rada, Janaćković, Đorđe, "Mg/Cu co-substituted hydroxyapatite – Biocompatibility, mechanical properties and antimicrobial activity" in Ceramics International, 45, no. 17, Part A (2019):22029, [https://doi.org/10.1016/j.ceramint.2019.07.219

Inteligentno stereo-vizuelno upravljanje mobilnih robota i optimalno terminiranje tehnoloških procesa - pregled rezultata istraživanja u okviru projekta MISSION4.0/Intelligent stereo-visual mobile robot control and optimal process planning and scheduling – overview of research results within the project MISSION4.0

Projekat MISSION4.0 podrazumevao je, u okviru nekoliko radnih paketa, razvoj inteligentnog stereo-vizuelnog upravljanja mobilnih robota, kao i optimalno planiranje i terminiranje tehnoloških procesa, i to baziranim na tehnikama veštačke inteligencije, posebno na konvolucionim veštačkim neuronskim mrežama i biološki inspirisanim algoritmima optimizacije. Tokom dvogodišnjih intenzivnih naučnih istraživanja razvijena je nova metodologija za autonomnu navigaciju i inteligentno upravljanje mobilnih robota sopstvenog razvoja, nazvanih RAICO i DOMINO. Generisanje optimalnog plana terminiranja tehnoloških procesa, u okviru koga se izvršava i inteligentni unutrašnji transport korišćenjem mobilnih robota, takođe je bio jedan od važnih ciljeva ovih naprednih istraživanja. U ovom radu, dat je pregled nekih od ključnih rezultata projekta MISSION4.0, poput publikovanih u vodećim međunarodnim i nacionalnim naučnim časopisima, objavljenih poglavlja u naučnim monografijama, saopštenih i odštampanih naučnih radova u zbornicima prestižnih konferencija održanih u inostranstvu i regionu, zatim u okviru verifikovanih tehničkih rešenja, kao i preko skupova podataka sa otvorenim pristupom

Inteligentno stereo-vizuelno upravljanje mobilnih robota i optimalno terminiranje tehnoloških procesa - pregled rezultata istraživanja u okviru projekta MISSION4.0/Intelligent stereo-visual mobile robot control and optimal process planning and scheduling – overview of research results within the project MISSION4.0

Projekat MISSION4.0 podrazumevao je, u okviru nekoliko radnih paketa, razvoj inteligentnog stereo-vizuelnog upravljanja mobilnih robota, kao i optimalno planiranje i terminiranje tehnoloških procesa, i to baziranim na tehnikama veštačke inteligencije, posebno na konvolucionim veštačkim neuronskim mrežama i biološki inspirisanim algoritmima optimizacije. Tokom dvogodišnjih intenzivnih naučnih istraživanja razvijena je nova metodologija za autonomnu navigaciju i inteligentno upravljanje mobilnih robota sopstvenog razvoja, nazvanih RAICO i DOMINO. Generisanje optimalnog plana terminiranja tehnoloških procesa, u okviru koga se izvršava i inteligentni unutrašnji transport korišćenjem mobilnih robota, takođe je bio jedan od važnih ciljeva ovih naprednih istraživanja. U ovom radu, dat je pregled nekih od ključnih rezultata projekta MISSION4.0, poput publikovanih u vodećim međunarodnim i nacionalnim naučnim časopisima, objavljenih poglavlja u naučnim monografijama, saopštenih i odštampanih naučnih radova u zbornicima prestižnih konferencija održanih u inostranstvu i regionu, zatim u okviru verifikovanih tehničkih rešenja, kao i preko skupova podataka sa otvorenim pristupom

Micro RNA-155 participates in re-activation of encephalitogenic T cells

MicroRNAs (miR) are small non-coding RNAs involved in the immune response regulation. miR-155 has been attributed a major pro-inflammatory role in the pathogenesis of multiple sclerosis and its animal model experimental autoimmune encephalomyelitis (EAE). Here, a role of miR-155 in re-activation of encephalitogenic CD4(+) T cells was investigated. Dark Agouti rats were immunized with myelin basic protein (MBP) emulsified in complete Freund's adjuvant. CD4(+) T cells were purified from draining lymph node cells (DLNC) obtained in the inductive phase and from spinal cord immune cells (SCIC) isolated at the peak of EAE. CD4(+) T cells obtained from SCIC (i.e., in vivo re-activated cells) had markedly higher expression of miR-155 in comparison to those purified from DLNC (not re-activated). Likewise, in vitro re-activation of DLNC with MBP led to increase in miR-155 expression. Further, DLNC and DLNC CD4(+) T cells were transfected with an inhibitor of miR-155 during in vitro re-activation. As a result, expression of important CD4(+) T cell effector cytokines IFN-gamma and IL-17, but not of regulatory cytokines IL-10 and TGF-beta, was reduced. These results imply that miR-155 supports re-activation of encephalitogenic CD4+ T cells. Our results contribute to a view that miR-155 might be a valuable target in multiple sclerosis therapy. (C) 2015 Elsevier Masson SAS. All rights reserved.Ministry of Education, Science and Technological Development of the Republic of Serbia {[}173035, 175038, 173013

Comparison of dendritic cells obtained from autoimmunty-prone and resistant rats.

Dendritic cells (DC) are responsible for the initiation and shaping of the adaptive immune response and are in the focus of autoimmunity research. We were interested in comparison of DC obtained from autoimmunity-prone Dark Agouti (DA) rats and autoimmunity-resistant Albino Oxford (AO) rats. DC were generated from bone marrow precursors and matured (mDC) by lipopolysaccharide. Tolerogenic DC (tolDC) obtained by vitamin D3 treatment were studied in parallel. Profile of cytokine production was different in AO and DA mDC and tolDC. Expression of MHC class II molecules and CD86 were higher in DA DC, while vitamin D3 reduced their expression in dendritic cells of both strains. Allogeneic proliferation of CD4+ T cells was reduced by AO tolDC, but not with DA tolDC in comparison to respective mDC. Finally, expression of various genes identified as differentially expressed in human mDC and tolDC was also analyzed in AO and DA DC. Again, AO and DA DC differed in the expression of the analyzed genes. To conclude, AO and DA DC differ in production of cytokines, expression of antigen presentation-related molecules and in regulation of CD4+ T proliferation. The difference is valuable for understanding the divergence of the strains in their susceptibility to autoimmunity